In a communications network such as the Optical Transport Network (OTN) defined in ITU-T Recommendation G.709 (G.709), which is hereby incorporated by reference herein, data is transmitted in a variety of types of client signals which are in turn transmitted in structured data units. G.709 specifies a number of Optical Data Unit (ODU) signals which all have the same frame format. One type of ODU signal is an ODUflex signal.
ODUflex signals use the same frame format as all ODU signals of the Optical Transport Network (OTN) defined in G.709. The payload carried by an ODUflex(GFP) signal is a stream of packets that have been encapsulated using the Generic Framing Procedure (GFP) protocol defined in ITU-T Recommendation G.7041 (G.7041), which is hereby incorporated by reference herein.
With reference to FIG. 1, which illustrates an example signal path 100 in the OTN, a source node 102 generates a client signal (such as, for example, an ODUflex(GFP) signal) which is transported through a plurality of intermediate nodes 104 (individually labeled 104-1 to 104-N) to a sink node 106 over a plurality of links 108. ODUflex(GFP) and other lower order signals are configured to be carried over links 108 between nodes in the OTN by High Order ODUk (HO ODUk) signals. A plurality of ODUflex signals (and other lower order signals) may be multiplexed into a single HO ODUk signal. A HO ODUk signal has a payload area (referred to as a HO Optical Payload Unit (HO OPU)) comprising one or more lower order signals, and an overhead area. At each intermediate node 104, HO ODUk signals received from an input link 108 are demultiplexed and the lower order signals are buffered before being multiplexed into HO ODUk signals output onto an output link 108. The payload area of a HO ODUk signal comprises a plurality of tributary slots (not shown in FIG. 1, see FIGS. 1A-1E), in which the lower order signals are sent. Each tributary slot has a rate of roughly 1.25 Gbit/s, although the exact rate depends on the rate of the HO ODUk signal. FIG. 1A shows an example tributary slot arrangement within a HO OPUk, where k=2 or 3. In general, the payload area of a HO OPUk is divided into n tributary slots arranged in a repeating pattern. Each tributary slot (TS) occupies one column of the payload area. FIG. 1B shows an example of a client signal using tributary slots 1, 2, 8 10 and 15.
ODUflex(GFP) signals are specified to have a normally constant bit rate selected to correspond to an integer multiple of the rate of a single tributary slot. There are 80 nominal ODUflex(GFP) signal rates defined in G.709, corresponding to the number of tributary slots that the ODUflex(GFP) signal occupies within the payload area of the High Order ODUk (HO ODUk) signal that carries it.
The Generic Multiplexing Procedure (GMP) used in the OTN provides a count of the number of data words to be transmitted in the next multi-frame (MF) of the HO ODUk signal. In this case, the data words being transmitted within the HO OPU (Optical Payload Unit) payload area channel is an ODUflex signal.
ITU-T Recommendation G.7044 (G.7044), which is hereby incorporated by reference herein, specifies a protocol for changing the rates of ODUflex(GFP) signals, referred to as the Hitless Adjustment of ODUflex(GFP) (HAO) protocol. The HAO protocol provides a mechanism for changing both the number of tributary slots occupied by an ODUflex(GFP) signal, and the rate of the of ODUflex(GFP) signal itself in order to appropriately fill that number of tributary slots. Changing the ODUflex(GFP) signal rate changes its capacity to carry the GFP-encapsulated client packet data frames. The HAO protocol consists of two parts. One part of the HAO protocol is a Link Capacity Resizing (LCR) protocol. LCR provides a per-link handshake mechanism that allows the nodes on each end of the link to synchronize a change in the number of tributary slots being used for a given ODUflex(GFP) signal. The synchronization allows the change in the number of tributary slots to occur with no impact on the ODUflex(GFP) client signal.
The other part of the HAO protocol is the bandwidth resizing (BWR) protocol. BWR changes the rate of the ODUflex(GFP) signal to fit within the new number of tributary slots. The ODUflex(GFP) signal rate change during BWR is achieved by changing the GMP count of the data words to be transmitted in the next multi-frame of the High Order ODU signal. The GMP mechanism thus allows a convenient mechanism for increasing or decreasing the ODUflex(GFP) signal rate. The rate change is spread out in time in order to allow all nodes along the path to adjust to the new rate gradually. In the case of a bandwidth increase, the LCR protocol completes before the BWR protocol begins. In other words, at the completion of the LCR protocol, the ODUflex(GFP) signal rate remains unchanged, but it occupies a different number of tributary slots. In a bandwidth decrease case, the order is partially reversed. The nodes on each link go through an initial LCR set-up so that they are prepared to pass the BWR signaling. Then the BWR protocol reduces the ODUflex(GFP) signal rate to fit the desired reduced number of tributary slots. Once the ODUflex(GFP) signal rate is appropriate for the smaller number of tributary slots, the LCR protocol completes the action by removing the tributary slots that are targeted for removal.
The HAO operation is illustrated in a summary manner in FIGS. 1C-E for a bandwidth increase case. FIGS. 1C-E show an example signal path 120 comprising a source node 122 which sends signals through a first link 123 to an intermediate node 124, which in turn sends signals through a second link 125 to a sink node 126. Each of first and second links 123 and 125 comprise a plurality of tributary slots 127.
An ODUflex client signal 130 (which may, for example, be an ODUflex(GFP) signal) is generated by a signal source 131 at source node 122, multiplexed into a HO ODUk signal by an interface 132 and transmitted through link 123 within four tributary slots 127 of the payload area of the HO ODUk signal. At intermediate node 124 the client signal 130 is then demultiplexed at interface 134 and passed through a switching fabric 135 to an interface 136 which multiplexes the client signal back into a HO ODUk signal for transmission over link 125. Intermediate node 124 comprises buffers (not shown) between switching fabric 135 and each of interfaces 134 and 136, and may also comprise other elements as known in the art, such as, for example, framer circuits (or “framers”) to recover signal alignment, multiplexers and demultiplexers, internal clocks, clock smoothing circuits (e.g. phase locked loops), and circuits to remove, insert and/or monitor the HO ODUk overhead information.
The LCR protocol is used on each link to change the number of tributary slots 127 used by the client signal 130. FIG. 1D illustrates that the LCR protocol is used to establish the connection for client signal through the two additional tributary slots 127, as indicated by the dashed double-headed arrows. The additional tributary slots 127 are used to transmit client signal 130 as its rate is increased during the BWR protocol. FIG. 1E shows the resulting use of all six tributary slots by an increased rate client signal 130′ after the BWR protocol operation.
The HAO protocol thus provides a convenient mechanism for changing the rate of an ODUflex client signal. However, as explained further below, depending on the number of intermediate nodes in a signal path and the type of changes to the rate of the ODUflex client signal, the nodes further down the signal path may experience problems due to over or under filling of their buffers. The inventors have identified a need for improved methods and systems for changing the rate of normally constant client signals such as ODUflex client signals.
There are a number of prior art proposals aimed at resolving this problem with the HAO protocol, which may be summarized as follows:
One proposal was to restrict the allowed number of intermediate nodes so that BWR can complete with reasonable stability in a reasonable timeframe. This proposal would have the advantage of not requiring any change in the relevant protocols, but would impose an additional network management burden, and may be overly limiting in terms of the total number of intermediate nodes allowed. Also, under such a proposal it would be difficult to specify the node requirements such that deterministic network behavior can be achieved.
Another proposal was to add signaling overhead to the HO OPU HAO overhead to indicate a step change in the rate and buffer stability at the transmitting node, and use the OPUflex overhead for the sink to inform the source of a stable connection that is now ready for the next step. This proposal has a number of advantages, in that it would provide guaranteed, deterministic network behavior, it would accommodate any number of intermediate nodes, and the signaling used to implement this proposal would remain at the layers already processed by source, sink, and intermediate nodes. However, this proposal would add more overhead than other approaches, would require that a “stability” determination at each node be defined, and may require more time to complete the BWR protocol as compared to other approaches. Also, this proposal would require a mechanism to communicate the step change event from the ingress line card to the egress line card at each node.
Another proposal was to add signaling overhead to the HO OPU HAO overhead to indicate a step change in the rate, and restrict the response of intermediate nodes. This approach would advantageously require only minimal new signaling overhead, and the signaling would remain at the layers already processed by source, sink, and intermediate nodes. However, this proposal would require a long enough time between rate steps for a meaningful random hold off, and network behavior would not be deterministic under this proposal. Also, this proposal would require a mechanism to communicate the step change event from the ingress line card to the egress line card at each node.
Another proposal was to add signaling to the OPUflex HAO overhead to indicate when a rate change step has been initiated by the source. Advantages of this proposal include fast flow-through of the step indication, which allows all intermediate nodes to adjust at virtually the same time, and minimization of the cascading/magnifying of the problem at subsequent nodes. However, this proposal would require intermediate nodes to process LO OPU overhead at their egress ports, and the nodes are still just reacting to the incoming signal rate. Also, further study would be needed to determine whether there could be problems with such an approach in a signal path with a large number of nodes.
Another proposal was to use normal mode filtering with to restrict the rate of rate change at each node to less than 300 steps per second, with a step size of one GMP increment (which is equal to one data word; e.g. N-bytes for an ODU signal using N tributary slots). This approach could result in some simplification to the HAO protocol by removing the GMP special mode filter change. However, this approach would increase the buffer requirements at each intermediate node (due to the filtering causing a slow egress response to the ingress rate change), and also may be slower than the other approaches. Also, this approach was found to not work during subsequent network simulations. In general, Phase Locked Loops (PLLs) can successfully track a frequency ramp with zero offset, but introduce a delay when given an abrupt start to a ramp, which can lead to the problem identified above.
The inventors have thus identified a need for improved methods and systems for changing the rate of normally constant bit rate client signals such as ODUflex client signals. The inventors have identified a particular need for improved methods and systems for changing the rate of the ODUflex(GFP) signal during the HAO protocol.
The HAO protocol is unique relative to other protocols for adjusting signal rates as known in the art. The most fundamentally unique aspect of HAO relative to other rate adjustment protocols is the existence of an intermediate bearer signal, which is the ODUflex(GFP) signal. Other rate adjustment protocols operate on the basis of packets being mapped (with or without some encapsulation protocol) directly into a Layer 1 channel. These protocols address either packet transmission rate of the data mapped into that channel, or the size of the Layer 1 channel. For example, the Link Capacity Adjustment Scheme (LCAS) defined in ITU-T Recommendation G.7042/Y.1305 (2001), which is hereby incorporated by reference herein, is used to change the size of a Layer 1 channel's capacity in a manner that causes no hits or disruptions to the packet data stream being carried within that channel. The LCAS is similar to the HAO LCR protocol, except that LCAS is more complex due to its other protocol requirements. Specifically, LCAS supports channels that consist of tributary slots on multiple different Layer 1 signals that can be routed on different Layer 1 paths through the network (similar to Link capacity adjustment within the IEEE 802.3 Link Aggregation (LAG) Protocol). However, an ODUflex signal requires that all the tributary slots be contained within the same signal on each link as it transits a network. There is no intermediate bearer signal with LCAS that would be analogous to the ODUflex(GFP). The GFP-encapsulated packets are mapped directly into the channel controlled by LCAS. The HAO protocol is different in that a nominally constant bit rate (CBR) OTN signal is defined (i.e., the ODUflex(GFP) signal), and the packets are first mapped into the payload portion of that intermediate signal. Then, this intermediate ODUflex(GFP) signal is time division multiplexed (TDM) into the payload area of a higher rate server signal (i.e., the HO ODUk). General packet stream rate adjustment protocols adjust the rate of packet transmission by simply varying the number of Idle characters sent between data packets. The LAG protocol allows an Ethernet stream to be divided and transmitted in parallel over multiple physical links, but does not provide a mechanism for changing the number of links in a hitless manner, nor does it allow an intermediate carrier (i.e., like an ODUflex signal) that is mapped into the links to smoothly and hitlessly ramp its rate. Also, the LAG protocol only affects how packets are mapped directly into the aggregate set of available links. No other protocol defines an intermediate container signal that is then multiplexed using TDM into a higher rate Layer 1 signal. The HAO protocol was developed as a mechanism to change the rate of this intermediate container signal to fill the capacity of a different sized Layer 1 channel (i.e., fill a different number of tributary slots within a HO ODUk server signal.)
There have been a number of prior art attempts at determining whether or not a node is behaving such that it will not cause problems for downstream nodes. The prior art attempted to specify the performance of nodes based on either (A) frequency domain methods (similar to the specifications for jitter and wander on a constant bit rate signal in ITU-T Recommendation G.8251 (G.8251), which is hereby incorporated by reference herein) or (B) by a bound on the changes in buffer fill at each node, or by a combination of these parameters.
(A) The jitter/wander type of approach has the following drawbacks:
While jitter and wander are relatively easy to measure as short-term phase/frequency variations of a fixed rate signal, it becomes much more complex to measure the equivalent aspects of a signal that is changing its rate. Here, the test equipment would need to measure the short term variations in the rate at which the ODUflex(GFP) signal rate is changing rather than in the ODUflex(GFP) signal rate itself.
There are scenarios in which a jitter/wander type approach would not guarantee stability. For example, the output rate change from a give node may be very stable, however if its rate change is too different from the next node (i.e., there is too much difference between the slope of the frequency change between the two nodes), the downstream node may still not be able to maintain its own output ramp stability without experiencing buffer underflow or overflow.
(B) The buffer stability type of approach has the following four drawbacks.
First, at a high level, buffer hysteresis is not necessarily relevant to the network performance. Specifically, a given node's buffer hysteresis does not directly impact the protocol unless that node reacts to the associated buffer fill in a manner that forces excessive buffer hysteresis at a downstream node.
Second, a buffer hysteresis approach is extremely implementation-dependent. For example, since many device and system implementations contain multiple buffers at different points in the datapath, the determination of the buffer hysteresis is not straightforward to either determine or specify.
The third problem is that, since buffer fill and buffer hysteresis are internal device/system states, it is impossible to test the system to verify compliance with a buffer-hysteresis specification. The method for specifying the stability of the ramp rate must be testable (and hence measurable) at the edges of a network element rather than requiring internal state knowledge of the network element.
The fourth issue with using hysteresis is that hysteresis is best used for a parameter that nominally has a steady state rather than a state that is changing. In order to understand this issue, consider the following example of a 1 km fiber. The amount data that resides within a fiber is the data rate divided by the light propagation rate, multiplied by the fiber length. When an ODUflex uses a single tributary slot, a 1 km fiber contains approximately (1.25×109 bit/s)(1 s/2×108 m)(1×103 m)=6250 bits. If that ODUflex rate is increased to using eight tributary slots, then that fiber would contain approximately 50000 bits. The increased number of bits contained in the fiber is inherent due to the fiber having a non-zero propagation delay, and it has no impact on the downstream equipment. A node is analogous to a piece of fiber in that the amount of data in its buffers for a given ODUflex signal is inherently proportional to the rate of that ODUflex signal. Consequently, a first order hysteresis specification is not directly applicable during the BWR protocol.
The inventors have thus identified a need for improved methods and systems for changing the rate of normally constant bit rate client signals such as ODUflex client signals wherein the performance of nodes may be tested to verify compliance with specified performance characteristics.